Likelihood Constrained Bi-Level Optimization of Classification Performance

 We wish to classify a high-dimensional observation as belonging to one of two classes. Toward that end, we present a bi-level optimization procedure for maximizing, conditional on observed training data and subject to maximum likelihood constraints, a measure of the -dimensional class-conditional probability density estimate separation for . The separation optimization involves traversal of a manifold of maximum likelihood solutions parameterized by a class of operators . This optimization is performed in pursuit of the ultimate objective: minimizing (over ) the probability of misclassification when using a Bayes classifier based on -dimensional maximum likelihood class-conditional mixture model density estimates. A penalty interior point approach to the required optimization is proposed which generates a solution that satisfies the fundamental first-order optimality conditions. The performance of the proposed algorithm is illustrated through the application to a multispectral minefield classification problem. Index Terms  Discrimination, dimensionality reduction, density estimation, mixture model, PIPA, MPEC, minefield. z R ∈ k k d < Pw:R d R → w W ∈ k This work was supported in part by Office of Naval Research Grant N00014-95-10777 (CEP), National Science Foundation Grant DMS-9705220 (CEP & TEO) National Science Foundation Grant CCR-9624018 (JSP), and Wright Patterson Air Force Base via Veda Contract F33615-94-D-1400 (CEP, JSP, & TEO). 2